According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

We'll likely have mypy 2.1 out pretty soon afterwards, so we can wait for 2.1.

Hm, our daily benchmark shows this PR caused 2% slow-down on self-check. Taking into account mypy itself doesn't use TypeForm at all, this is not acceptable.

I guess the slow-down is because we call try_parse_as_type_expression() too much, which btw makes two re.match() calls that may be quite expensive (because they will call into interpreted code).

I think we should only interpret an expression as type whenever a TypeForm is expected. I don't think we need to revert this yet as it is not included in 2.0, but the performance should be addressed before 2.1 (i.e. soon).

I’ll take a look at the performance on selfcheck in the next few days.

The TypeForm-related code already has “cache-hit” / “cache-miss” style performance counters plus a script to interpret them, which should be useful for investigation.

Btw, I briefly looked at the current TypeForm logic. Initially I thought the try_parse_as_type_expression() calls during semantic analysis are some kind of performance optimization (that went wrong), but it looks like they are required because of the TypeVarLikeScope issue in type checker. @davidfstr is this right?

I don't know how hard it is, but I think the optimal way would be to fix the TypeVarLikeScope thing, and always parse expressions as types lazily. If this is too hard, some ideas to avoid too much extra work in semantic analysis:

• re calls are actually slow (because they are not optimized by mypyc). So maybe before calling re filter some common cases, like skip empty strings, and strings of length 1 if it is not isalpha().
• Skip parsing when assigning to an inferred variable
• Skip parsing when assigning to a variable with a type that is known to not have TypeForm
• Skip parsing return expression when return type annotation is missing of doesn't have TypeForm
• Skip parsing arguments when calling a known function without TypeForm in parameter types

For the last items you don't need to do an extra visit, simply record the presence of TypeForm in type analyzer. But if something like type TypeList[T] = list[TypeForm[T]] is a valid thing per the PEP, then you will need to be careful when recording the presence of type forms (you may need to add an attribute to TypeAlias node).

Don't know how much it will help but I noticed an easy opportunity to get rid of one of the two regexes: #21459. Also looking into the second one a bit to see if we can get a better heuristic.

Rearranged the code a bit more based on some observations about strings appearing in mypy itself.

If the regression is really driven by the string matching in try_parse_as_type_expression (haven't profiled to verify), then this is going to be pretty application-dependent. Mypy might be a slightly pathological case because there are a lot of strings like "builtins.list" that look to the regex like they could be valid type expressions.

If the regression is really driven by the string matching

To be clear, I don't think it is the main factor, but likely a significant factor. I think it may be necessary to do most of the things I mentioned above to make the regression go away ~completely.

Initially I thought the try_parse_as_type_expression() calls during semantic analysis are some kind of performance optimization (that went wrong), but it looks like they are required because of the TypeVarLikeScope issue in type checker. @davidfstr is this right?

IIRC the only reason there’s logic in the semantic analyzer (and not just only in the TypeChecker) is because string literals are sometimes type annotations and TypeChecker doesn’t have enough info to resolve all references that could occur in a string context. In particular locally defined classes are thrown out by the time TypeChecker gets to look at an expression.

I’m still tracking the task of optimizing performance related to this thread. Last week I was finishing up items for work as I was transitioning out into parental leave. Hopefully I should have some more bandwidth over the next few days.

Mypy might be a slightly pathological case because there are a lot of strings like "builtins.list" that look to the regex like they could be valid type expressions.

Btw if someone can benchmark this PR on something unrelated, like homeassistant or torch (the repo itself, not as 3rd party import), then we may get a better idea of the impact. If the impact on unrelated repos is <= 1% then I think we don't need to revert this for v2.1 (we will still need to profile/optimize this however).